Greg Detre
Tuesday, October 29, 2002
Gardenfors
wants to add a new cognitive �level� or �perspective� sandwiched between the
connectionist and the symbolic, which he terms the �conceptual�. In short, the
conceptual level can be seen as a low-dimensional vector space, where each
dimension is interpretable as a �quality�, like pitch, or one of the four
(five?) taste receptors, or some other more abstract �[representation] used as
a modelling factor in describing mental activities of organisms�.
This
introduction of the conceptual level follows a useful summary of the debate
between supporters of the symbolic and connectionist paradigms, which can be
seen as making most of the same points found in Harnad�s �The Symbol Grounding
Problem�. These include: symbols are meaningless/ungrounded in the real world;
symbolic systems are static and cannot learn/self-organise unsupervised; and
connectionist systems may get the job done but might be uninterpretable. The conceptual
level is intended, it seems, to address these difficulties from both
directions.
However,
although I find Gardenfors� idea interesting, I�m unclear as to exactly what
the proposal amounts to. I think he wants it to be a level in just the sense that
the symbolic and connectionist are levels. That is, at each level, we can
discard all the granularity/noisiness that is unnecessary for computations at
that level. At the connectionist level, we ignore the atomic, molecular and
cellular (say) details that underly the computations in our neural network, and
talk simply in terms of activation, synapse strengths etc. At the symbolic
level, we ignore the implementation details (whether connectionist, silicon
etc.), and talk at the abstract level of symbols, rules etc. Apart from it
being a low-dimensional vector space that �emerges� from the connectionist
level, I don�t feel that we know enough about the conceptual level to
understand exactly the sort of computations and lossy abstractions being
performed to get from the connectionist, or indeed up to the symbolic, from the
conceptual.
Let us
consider how, in principle, the move from the connectionist all the way up to
the symbolic is supposed to happen. There, we would like to be able to assume
that the symbolic information is somehow inherent in the overall behaviour of
the system, even if there are no single, discrete parts of the system that
represent each symbol explicitly. Presumably, the same is roughly true of the
conceptual level � although the quality-dimensions are not explictly contained
in the activity of any one neuron, they might be spread over a population of
neurons, or indeed somehow contained implicitly in the whole system. But if
this is the case, then I don�t see how the introduction of the conceptual level
has helped us bridge the gap between connectionist and symbolic at all. In
fact, it feels as though the conceptual can be reduced easily to a special case
of the symbolic � the quality-dimensions that Gardenfors considers are all
paradigmatic symbols. We could much more naturally think of taste as a function
of the four/five taste receptor symbol values, colour as a function of three
quality-dimensions, time as a number-line symbolic value, kinship relations as
a symbolic hierarchy etc. The traditional problem with grounding the symbolic,
it seems to me, has been the difficulty of achieving this seamless continuum
across levels that he claims is rendered effortless by the insertion of a
sandwich level. If we can ground the quality-dimensions, paradigmatic symbols
that they are, then we can also ground the symbolic (or at least some of it).
Even more
importantly, I think we would have real difficulty mapping most of our concepts
onto quality-dimensions. Concepts like smell, higher-level visual or auditory
concepts and most lexical concepts (just to name a few) would all be too fuzzy
and high-dimensional to usefully quantify in low-dimensional terms. Indeed, any
such mapping would probably be more abstract than the symbolic mapping
that is supposed to sit on top. Perhaps we should focus instead on his
suggestion that the conceptual level allows us to employ a frame-like model in
our cognitive model as a means of discretising context, but I think this brings
its own problems too.
Finally,
given that the conceptual lacks the fine-grained computational tools of
connectionist representations, and the high-level syntactic rules of the
symbolic, it doesn�t seem to help us at all when it comes to understanding how
high-level representations could self-organise/learn. I think Gardenfors�
problem is that he has reduced the dimensionality too far in his description of
the conceptual level. Perhaps though there is space for a sandwich level lower
down than Gardenfors places the conceptual.
Hofstadter�s
review of Holyoak and Thagard is pretty damning, and seemingly with good cause.
Their strategy is to encode very restricted fragments of real-world knowledge
into predicate calculus, and to model analogising as seeing correspondence in
the structure of this encoding.
Hofstadter
attacks them on these and further grounds:
a)
The
fragments they encode are tiny and appear to contain only highly relevant facts
� �tiny, hand-sculpted, frozen caricatures�
b)
Using
predicate calculus effectively presumes the analogy being sought, partly by
nature of the encoding, and partly by the propositions chosen � �they do not
seem to realise the hugeness of the gulf between a full situation as a human
perceives it � having no sharp boundaries, woven intricately into the fabric of
one�s knowledge and life experiences � and a handful of predicate calculus
formulas�
c)
They
emphasise one-to-one correspondences between the elements of analogous
situations. This is exemplified by Holyoak and Thagard�s example of the analogy
between Hitler�s Germany and Hussein�s Iraq that could easily be extended to
orchestras, and an enormous variety of predicates that happen to have the same
two-place structure.
d)
They
see causal relationships as a central anological connection at the expense of
seeing the complex situational themes present in the gist � �such an overstress
on causality � a ubiquitous ingredient of events and therefore, an essentially
useless tool for classifiying them � would allow virtually any pair of events
to map onto each other�
The
resulting �analogies� that are found would not capture all and only the
analogies that we (humans) find interesting. This is ultimately the acid test
of a model of analogy-making. ACME is able to appear to insightful despite its
rudimentary processing because if its knowledge-base was extended to huge
swathes of the real world, it would see analogies everywhere.
I�m not
sure how far I agree with his declaration that �gist extraction, the
ability to see to the core of the matter, is the key to analogy-making�. I
think my uncertainty is probably a result of not having read enough of his new
work to feel clear about what he means by the �gist�. I feel as though he could
mean two things by �gist extraction�:
a)
being
able to abstract down to a brief narrative or essential summary of some
fragment of knowledge
b)
being
able to reconsider that knowledge from different perspectives or at different
levels
I would
like to say that it is the second reading that is most important in
analogy-making, and which seems to be the approach that Hofstadter employs in
analogising throughout most of the review, while it is the first reading which
is especially �key � to all intelligence�. However, I don�t want to push this
distinction, because it could be that the two readings could be seen as more or
less the same thing. After all, if one can reconsider some fragment of
knowledge at a high level from various perspectives, then presumably one could
provide various narratives or summaries of it, and vice versa.
Regier and
Carlson are seeking to understand how our perceptual processes and our
linguistic description of relations based on those perceptual processes are
linked together. They analyse the projective notion of �above�, comparing 4
different computational models to empirical findings from 7 experiments
designed to show which model best fits our intuitions about the notion �above�,
measured in terms of a human �acceptability� rating for various relational
positions of a landmark shape and trajector point.
The four
models were:
1.
Bounding
box model � �a
trajector object is above a landmark object if it is higher than the highest
point of the landmark and between its rightmost and leftmost points�
2.
Proximal
and Centre-of-mass model � defines aboveness in terms of polar coordinates, i.e. angular
deviation from upright. This can either be measured from the ray passing
through the horizontal centre of the landmark (centre of mass) or the
orientation of the ray connecting the closest edge of the landmark to the
closest edge of the trajector (proximal)
3.
Hybrid
model �� this is a variant of the
proximal/centre-of-mass model that also takes into account how much higher the
trajector is than the landmark
4.
Attentional
vector sum model �
this is informed by two independent observations:
a.
human
apprehension of spatial relations involves attention
b.
overall
direction is often neurally represented as the vector sum of a set of
constituent directions
The AVS model defines aboveness in terms of a population of vectors,
from each point in the landmark, as a function of the polar coordinate and an
exponential drop-off away from the point at the top of the landmark vertically
aligned with the trajector.
When the exponential attentional drop-off from the vertically-aligned
point is high, (narrow beam) the AVS model approaches the proximal
orientation. When the exponential drop-off is low, (wide beam) the AVS
model reduces to the centre-of-mass orientation.
They find
that the AVS model best fits the experimental data.
Put
pithily, Regier & Carlson are talking about concepts of geometry, whereas
Gardenfors is talking about a geometry of concepts.
We can see
Regier & Carlson�s model of spatial semantics as a cognitive domain that we
can try to fit into Gardenfors� three-level hierarchy. Presumably, the Regier
& Carlson model would fit in somewhere above the connectionist, since it�s
expressed in higher-level, lower-granularity, noiseless terms. The notion of
�above� makes a good example for probing the Gardenfors system, because it fits
all of Gardenfors� criteria for a quality-dimension nicely, yet shows why the
idea of a quality-dimension is confusing. On the one hand, you�re unlikely to
find a better candidate for modelling concepts in geometric terms than actual spatial/geometric
concepts (like above), since they consist of a small number of orthogonal
magnitudes that we can effortless map onto quality-dimensions at the conceptual
level. However, at the same time, Regier & Carlson�s model could equally
well be seen as at the symbolic level, since �aboveness� is expressed as a
symbolic expression (a mathematic formula). It seems as though the conceptual
level is merely a slightly looser subset of the symbolic level, since we still
have no clearer idea of how to ground it than we do for grounding anything else
at the symbolic level. Indeed, Regier & Carlson are loath to consider their
hypotheses as hypotheses about actual neural mechanisms.
I found it difficult to see where the syllogism is clashing with my notion of �reasonableness�. Clearly, the conclusion appears paradoxical, but pinpointing the bait-and-switch is tricky.
One option would be simply to reject the second premise. This is certainly a possibility with the first syllogism, where we might want to deny that �all rare things are expensive� � in this case, the syllogism could be sound, but the truth-preserved conclusion would still be false. As a result, I felt much happier looking at the second syllogism.
Its tighter language precluded any obvious ambiguities, either in the words �rare� or �cheap�. �Rare� could be taken to mean either �there aren�t many of these� or more strongly, �there aren�t many of these and they�re desirable�. Analogously, the word �cheap�, which could be taken to mean �cheap and nasty�, or could be taken to mean �costing less than you�d expect for something desirable�. Similarly, the caveat �in Boston� presumably allows us to rule out any possibility that rarity could be localised due to an uneven distribution. I don�t think either of these problems get to the root of the problem, but it�s nice to be able to discard them.
A different
approach I tried involved trying to understand what it is about the particular
combination of predicates chosen here that makes the example work. For
instance, the classic syllogism:
Socrates is a man
All men are mortal
\ Socrates is mortal
seems to
take a pretty similar form, yet obviously is not erroneous. I found that if I
swapped the predicate �is expensive� for �has a propensity to rise in price�,
we get:
Cheap apartments (in Boston that are desirable)
are rare (things)
Rare things (in Boston that are desirable) have a propensity to rise in price
\ Cheap apartments (in Boston that are
desirable) have a propensity to rise in price
This
doesn�t seem at all unreasonable. The problem is that I�m not sure that this
use of �expensive� says exactly the same thing as the original. But it did seem
to indicate that there�s something odd about the transitoriness of our notion
of �price�. Somehow, what we want to say is that cheap apartments are rare because
they�re not expensive, since after all price is a function of
desirability and rarity, i.e.
Desirable things are generally expensive
Things that should be expensive but are cheap
are rare.
\ Desirable things that are cheap are rare
This use of
generally changes things. For starters, I think it must take us out of the
bounds of what can be expressed using first order logic, since it amounts to a
�mostly� quantification, i.e. somewhere in between �forAll� and �thereExists�.
Finally, I briefly considered that part of the problem might be that the predicates being discussed are extrinsic properties that can be attached/detached from an object without affecting its intrinsic meaning, but couldn�t really see how to make this stick.
In short, I�m not sure which, if any, of these is the source of the paradox in the conclusion.